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Method and Apparatus for Building Multimedia Applications Using 

Interactive Multimedia Viewers 

by Inventors 
Christopher Clemmett Macleod Beck et ah 

Field of the Invention 

The present invention is in the field of telecommunication 
encompassing all existing sorts of interaction multimedia technology, and 
pertains more particularly to methods and apparatus for providing a 
seamless viewing and media embedding capability for various types of 
media interactions that may occur in a multimedia call-center operating a 
customer-interaction-network-operating-system (CINOS). 

Cross-reference to related documents 

The present application is a continuation-in-part (CIP) of copending 
application P3315PA, which is a CIP of copending application P3314PA 
and P3315PA, all of which are incorporated herein in their entirety by 
reference. 



Background of the Invention 



In the field of telephony communication, there have been many 
improvements in technology over the years that have contributed to more 
efficient use of telephone communication within hosted call-center 
environments. Most of these improvements involve integrating the 
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telephones and switching systems in such call centers with computer 
hardware and software adapted for, among other things, better routing of 
telephone calls, faster delivery of telephone calls and associated 
information, and improved service with regard to client satisfaction. Such 
5 computer-enhanced telephony is known in the art as computer-telephony 
integration (CTI). 

Generally speaking, CTI implementations of various design and 
purpose are implemented both within individual call-centers and, in some 
cases, at the telephone network level. For example, processors running CTI 

10 software applications may be linked to telephone switches, service control 
points (SCPs), and network entry points within a public or private telephone 
network. At the call-center level, CTI-enhanced processors, data servers, 
transaction servers, and the like, are linked to telephone switches and, in 
some cases, to similar CTI hardware at the network level, often by a 

15 dedicated digital link. CTI processors and other hardware within a call- 
center is commonly referred to as customer premises equipment (CPE). It is 
the CTI processor and application software is such centers that provides 
computer enhancement to a call center. 

In a CTI-enhanced call center, telephones at agent stations are 

20 connected to a central telephony switching apparatus, such as an automatic 
call distributor (ACD) switch or a private branch exchange (PBX). The 
agent stations may also be equipped with computer terminals such as 
personal computer/video display unit's (PC/VDU's) so that agents manning 
such stations may have access to stored data as well as being linked to 

25 incoming callers by telephone equipment. Such stations may be 

interconnected through the PC/VDUs by a local area network (LAN). One 
or more data or transaction servers may also be connected to the LAN that 
interconnects agent stations. The LAN is, in turn, typically connected to the 
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CTI processor, which is connected to the call switching apparatus of the call 
center. 

When a call arrives at a call center, whether or not the call has been 
pre-processed at an SCP, typically at least the telephone number of the 
5 calling line is made available to the receiving switch at the call center by the 
network provider. This service is available by most networks as caller-ID 
information in one of several formats such as Automatic Number 
Identification (ANI). Typically the number called is also available through 
a service such as Dialed Number Identification Service (DNIS). If the call 

10 center is computer-enhanced (CTI), the phone number of the calling party 
may be used as a key to access additional information from a customer 
information system (CIS) database at a server on the network that connects 
the agent workstations. In this manner information pertinent to a call may 
be provided to an agent, often as a screen pop on the agent's PC/VDU. 

15 In recent years, advances in computer technology, telephony equipment, 

and infrastructure have provided many opportunities for improving telephone 
service in publicly-switched and private telephone intelligent networks. Similarly, 
development of a separate information and data network known as the Internet, 
together with advances in computer hardware and software have led to a new 

20 multimedia telephone system known in the art by several names. In this new 

systemology, telephone calls are simulated by multimedia computer equipment, 
and data, such as audio data, is transmitted over data networks as data packets. In 
this system the broad term used to describe such computer-simulated telephony is 
Data Network Telephony (DNT). 

25 For purposes of nomenclature and definition, the inventors wish to 

distinguish clearly between what might be called conventional telephony, which is 
the telephone service enjoyed by nearly all citizens through local telephone 
companies and several long-distance telephone network providers, and what has 
been described herein as computer-simulated telephony or data-network 
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telephony. The conventional systems are referred to herein as Connection- 
Oriented Switched-Telephony (COST) systems, CTI enhanced or not. 

The computer-simulated, or DNT systems are familiar to those who use 
and understand computers and data-network systems. Perhaps the best example 
5 of DNT is telephone service provided over the Internet, which will be referred to 
herein as Internet Protocol Network Telephony (IPNT), by far the most extensive, 
but still a subset of DNT. 

Both systems use signals transmitted over network links. In fact, 
connection to data networks for DNT such as IPNT is typically accomplished over 

10 local telephone lines, used to reach points in the network such as an Internet 

Service Provider (ISP). The definitive difference is that COST telephony may be 
considered to be connection-oriented telephony. In the COST system, calls are 
placed and connected by a specific dedicated path, and the connection path is 
maintained over the time of the call. Bandwidth is basically assured. Other calls 

15 and data do not share a connected channel path in a COST system. A DNT 

system, on the other hand, is not dedicated or connection-oriented. That is, data, 
including audio data, is prepared, sent, and received as data packets over a data- 
network. The data packets share network links, and may travel by varied and 
variable paths. 

20 Recent improvements to available technologies associated with the 

transmission and reception of data packets during real-time DNT 
communication have enabled companies to successfully add DNT, 
principally IPNT, capabilities to existing CTI call centers. Such 
improvements, as described herein and known to the inventor, include 

25 methods for guaranteeing available bandwidth or quality of service (QoS) 
for a transaction, improved mechanisms for organizing, coding, 
compressing, and carrying data more efficiently using less bandwidth, and 
methods and apparatus for intelligently replacing lost data via using voice 
supplementation methods and enhanced buffering capabilities. 
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In addition to Internet protocol (IPNT) calls, a DNT center may also 
share other forms of media with customers accessing the system through 
their computers. E-mails, Video mails, fax, file share, file transfer, video 
calls, and so forth are some of the other forms of media which may be used. 
5 This capability of handling varied media leads to the term multimedia 

communications center. A multimedia communications center may be a 
combination CTI and DNT center, or may be a DNT center capable of 
receiving COST calls and converting them to a digital DNT format. The 
term communication center will replace the term call center hereinafter in 

10 this specification when referring to multimedia capabilities. 

In typical communication centers, DNT is accomplished by Internet 
connection and IPNT calls. For this reason, IPNT and the Internet will be 
used in examples to follow. IT should be understood, however, that this 
usage is exemplary, and not limiting. 

15 In systems known to the inventors, incoming IPNT calls are 

processed and routed within an IPNT-capable communication center in 
much the same way as COST calls are routed in a CTI-enhanced call-center, 
using similar or identical routing rules, waiting queues, and so on, aside 
from the fact that there are two separate networks involved. Communication 

20 centers having both CTI and IPNT capability utilize LAN-connected agent- 
stations with each station having a telephony-switch-connected headset or 
phone, and a PC connected, in most cases via LAN, to the network carrying 
the IPNT calls. Therefore, in most cases, IPNT calls are routed to the 
agent's PC while conventional telephony calls are routed to the agent's 

25 conventional telephone or headset. Typically separate lines and equipment 
must be implemented for each type of call weather COST or IPNT. 

Due in part to added costs associated with additional equipment, 
lines, and data ports that are needed to add IPNT capability to a CTI- 
enhanced call-center, companies are currently experimenting with various 
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forms of integration between the older COST system and the newer IPNT 
system. For example, by enhancing data servers, interactive voice response 
units (IVR's), agent-connecting networks, and so on, with the capability of 
conforming to Internet protocol, call data arriving from either network may 
5 be integrated requiring less equipment and lines to facilitate processing, 
storage, and transfer of data. 

With many new communication products supporting various media 
types available to businesses and customers, a communication center must 
add significant application software to accommodate the diversity. For 

10 example, e-mail programs have differing parameters than do IP applications. 
IP applications are different regarding protocol than COST calls, and so on. 
Separate routing systems and/or software components are needed for routing 
e-mails, IP calls, COST calls, file sharing, etc. Agents must then be trained 
in the use of a variety of applications supporting the different types of 

15 media. 

Keeping contact histories, reporting statistics, creating routing rules 
and the like becomes more complex as newer types of media are added to 
communication center capability. Additional hardware implementations 
such as servers, processors, etc. are generally required to aid full multimedia 

20 communication and reporting. Therefore, it is desirable that interactions of 
all multimedia sorts be analyzed, recorded, and routed according to 
enterprise (business) rules in a manner that provides seamless integration 
between media types and application types, thereby allowing agents to 
respond intelligently and efficiently to customer queries and problems. 

25 One problematic issue that is inherent in current art communication 

centers is that standardization of media viewing capability with respect to 
agents, business contacts, and customers is virtually happenstance. For 
example, in order to interact with agents at a current-art communication 
center, a customer must be operating the proper system with the proper 
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platform that contains the appropriate software applications and application 
program interfaces (API's) that are also supported and used at the 
communication center. For instance, an agent may be using one type of 
word processing application, while the customer prefers a different word 
5 processing application and so on, and the difference may hinder 

communication. In many instances, complicated conversions may have to 
be performed on both sides. The same issue applies to many graphics 
programs, video applications, and even IP applications. 

In many instances, customers or contacts are operating different 

10 operating platforms or even computer systems i.e. Macintosh vs. IBM. 

Often such customers or contacts must be segregated from the majority so 
that they may receive the proper software, that their platform or system 
supports before they may interact with an agent at the communication 
center. In most cases their functionality will be limited because often 

15 special versions of certain applications may not be not available. 

Furthermore, agents may desire to perform certain calculations or 
conduct research that may be directed to streamlining or improving 
performance. A narrow example of such a researched calculation may be a 
desire to determine the communication center's top 500 customers in terms 

20 of profitability. Such research may be time-consuming and difficult at best, 
especially in a multimedia communication center. Such communication 
centers may have many databases wherein variant types of media may be 
stored. An agent may be required to access media records wherein differing 
types of programs are required to disseminate the media. 

25 Many organizations offer downloadable software applications 

termed Viewers in the art, that enable customers or clients to view a 
particular type of media or format that otherwise the customer or client 
would not be able to access and use. A drawback to these types of viewers 
is that they typically support only a few different types of applications at the 
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most. Therefore, many viewers must be obtained in order to view several 
differing media types or view media from differing platforms. It is desirable 
then, to enable designed and dedicated interaction applications adapted to 
perform certain functions according to enterprise rules. Such applications 
5 dedicated to performing predefined processes may be created and made 
available through a standard application format to help guide customers, 
contacts and agents through such processes as described above. 

What is clearly needed is a method and apparatus for building and 
providing interactive applications through which customers, clients, agents, 
10 business partners and the like may disseminate and view certain material in 
an application in a media-independent fashion according to a structured and 
predefined process. 

Summary of the Invention 

15 

In a preferred embodiment of the present invention an object- 
oriented programming interface for use by a programmer, a software 
Interactive Media Viewer (IMV) module is provided, comprising a code set 
adapted to access and render media code from multimedia files stored in a 

20 data repository; and an editable layer allowing the programmer to program 
limitations limiting access by the IMV to preselected media files. The IMV 
further comprises one or more software interfaces to other software modules 
that may be grouped in an Interactive Media Application (IMA) with one or 
more IMVs, may be adapted to access and render multimedia code of just 

25 one type or more than one type. 

In a preferred embodiment the multimedia files stored in the data 
repository represent multimedia transactions, and are characterized with tags 
according to one or more of date, time, participants, file type, company 
affiliation of participants, subject or issue, and relationship to other 
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multimedia files. In this embodiment IMVs are limited through the editable 
layer to tags of the multimedia files. 

Unique viewers are disclosed and taught according to the limitations 
above, and a multimedia communications center utilizing such viewers is 
5 taught as well. 

In another aspect of the invention, in a Multimedia Communication 
Center environment which includes access to and rendering of multimedia 
files stored in a data repository, a method is provided for assembling an 
Interactive Multimedia Application (IMA), comprising steps of (a) selecting 

10 first selectable software modules providing functionality for an Interactive 
Multimedia Application other than access to and rendering of the 
multimedia files; (b) selecting at least one selectable Interactive Multimedia 
Viewer (IMV) software module including a code set adapted to access and 
render media code from multimedia files stored in a data repository and an 

15 editable layer allowing the programmer to program limitations limiting 

access by the IMV to preselected media files; (c) editing the editable layer 
of the at least one IMV; and (d) joining the selected and edited modules to 
form the IMA. 

In these disclosures, for the first time, a system is provided for 
20 quickly and efficiently creating applications for using a multimedia 

communication center wherein media viewer modules are provided as 
editable modules wherein access to stored files may be effectively 
controlled. 



25 
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Brief Description of the Drawing Figures 

Fig. 1 is a diagram of a multimedia communications center enhanced 
with a network operating system according to an embodiment of the present 
5 invention. 

Fig. 2 is a block diagram illustrating basic layers of a customer 
interaction operating system according to an embodiment of the present 
invention. 

Fig. 3 is a flow chart illustrating basic steps performed by the 
10 network operating system of Fig. 2 related to completing interactive 
transactions between business partners. 

Fig. 4 is a block diagram illustrating agent-desktop function 
according to an embodiment of the present invention. 

Fig. 5 is a block diagram of an exemplary WEB-form customer 
15 interface according to an embodiment of the present invention. 

Fig. 6 is a flow chart illustrating media-presentation and customer- 
interface logic steps according to an embodiment of the present invention. 

Fig. 7 is an exemplary overview of a multimedia interaction storage 
system within a communication center according to an embodiment of the 
20 present invention. 

Fig. 8 is a block diagram of the repository of Fig. 7 illustrating 
threaded text-blocks and their relationship to stored multimedia according to 
an embodiment of the present invention. 

Fig. 9 is a process flow chart illustrating logical steps taken when 
25 building a threaded multimedia contact-history of communication-center 
interactions according to an embodiment of the present invention. 

Fig. 10 is a block diagram illustrating an interactive multimedia 
application (IMA) tool kit and a created application according to an 
embodiment of the present invention. 
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Fig. 1 1 is a process flowchart illustrating logical steps for building 
an IMA for a user interacting with CINOS according to an embodiment of 
the present invention. 

Description of the Preferred Embodiments 

Fig. 1 is a multimedia communications center enhanced with a 
network operating system according to an embodiment of the present 
invention. A telephony-network architecture 1 1 comprises an enterprise- 
hosted communication center 17 that is linked to, in this example, both a 
publicly-switched telephone network (PSTN) 13, and a wide area network 
(WAN) 15, which may be the public Internet or other digital network, such 
as a company Intranet. 

In this particular embodiment communication center 17 handles both 
conventional telephone calls, which may be categorized as connection 
oriented switched telephony (COST) calls, and data network telephony 
(DNT) calls, which may be DNT calls over a private digital network or calls 
according to a protocol such as the well-known Internet protocol. DNT 
calls are characterized in that data is transmitted as addressed data packets 
as opposed to dedicated connections in COST calls. As indicated, PSTN 13 
may be a private rather than a public network. WAN 15 may be a company 
Intranet, the Internet, or another type of WAN known in the art. The 
particular method of call delivery and call center integration is not 
particularly relevant for the purposes of this invention. There are many ways 
known both to the inventor as well as known in the art. Particular issues 
discussed in the disclosure between the telephones and the computers might 
be implemented differently depending on the actual system, but shall be 
deemed equivalent for all purposes of this invention. 
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Incoming COST calls arrive at a network-level telephony switching 
apparatus 19 in network cloud 13 and are connected over trunk 23 to a 
central telephony switching apparatus 27 within communication center 17. 
From switching apparatus 27, calls are routed according to existing routing 
5 rules over internal wiring 56 to agents' telephones 47, 49, 51, and 53 
residing at agents' workstations 31, 33, 35, and 37 respectively. 

Incoming DNT calls, and other communication events such as e- 
mail, file transfers and the like, arrive at a routing node 21 in WAN 15 and 
are passed on over digital connection 25 to a routing server 29 within 
10 communication center 17. Once calls arrive at server 29, they may, in some 
embodiments, be routed directly over LAN 55 according to existing routing 
rules to personal computer/video display units (PC/VDU) such as PC/VDU 
39, 41, 43, or 45 located at agent's workstations 31, 33, 35, and 37 
respectively. 

15 In this embodiment, switch-connected telephones 47-53 are also 

connected to PC/VDU' s 39-45 via a headset to computer sound-card 
according to technique known to the inventor and accomplished via an I/O 
cable. Thus connected, agents may respond to incoming COST and DNT 
calls with the same headset. 

20 hi the exemplary system and communication center shown, the 

equipment and applications are adapted to provide for multimedia operation 
at each of the agent stations, so the agents can interact with clients in many 
different ways, as are known in the multimedia arts. 

Computer telephony integration (CTI) enhancement is, in this 

25 embodiment, provided both at communication center 17 and in PSTN 13. 
For example, in PSTN 13, a processor 61 running instances of a CTI 
application known as a T-server (TS) to the inventors, and a statistics server 
(Stat) is connected to telephony switch 19 via CTI link 65. An intelligent 
peripheral 59 of the form of an interactive voice response unit (IVR) is 
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connected to processor 61 via data connection 63. Similar CTI equipment is 
illustrated within communication center 17. Namely, a processor 67 
running instances of TS and Stat and connected to telephony switch 27 via 
CTI link 71, and an IVR 69 connected to processor 67 via a data connection 
5 73, with processor 67 further connected to a local area network (LAN) 55 
within communication center 17. 

In alternative embodiments there may also be a CTI processor 22 in 
WAN 15 connected to server 21 by a CTI link 24. Also in some 
embodiments a separate data network 66 connects these CTI processors. In 

10 this way, intelligent routing may be performed at the network level with 
negotiation and direction from within communication center 17. 

It will be appreciated by those with skill in the art that the CTI 
enhancements, as immediately described above, may be hosted on one 
processor at PSTN 13 and on one processor at communication center 17 

15 without departing from the spirit and scope of the present invention. The 
inventor has chosen to show separate processors having separate functions 
for exemplary purposes only. It will also be appreciated by the skilled 
artisan that there may be many more or fewer than the four agent stations 
shown in communications center 17, and hardware and software 

20 arrangements may be made is a variety of ways. Also, home agents might be 
connected in a variety of ways to the call center. 

In a preferred embodiment of the present invention, a customer- 
interaction network operating system, hereinafter termed (CINOS), is 
provided for the purpose of managing communications center 17, and 

25 optimizing and recording all agent/customer interactions received at 

communication center 17 from networks 13 and 15. CINOS is unique in the 
fact that it is a multi-tiered object-and process-orientated system wherein 
logic regarding the various aspects of it's functionality is achieved via 
knowledge-based architecture and object modeling. Various functions of 
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CINOS, more fully described below, include capturing (recording), 
analyzing, routing, and, in many instances, responding via automated 
process to customers engaged in interactions with the enterprise (company 
hosting the communication center). CINOS is adapted to support all 
5 planned communication mediums such as multimedia DNT applications 
including e-mail, video mail, file transfers, chat sessions, IP calls, and CTI 
COST transactions such as voice calls, voice mails, faxes, and so on. 

Referring back to Fig. 1, CINOS utilizes various LAN-connected 
machines in order to perform various operations. Among these various 

10 hardware implementations are a multimedia server (MIS) 79 adapted to 
physically store and serve all multimedia transactions, and a customer- 
information-system server (CIS) 57 adapted to physically store and serve 
information relevant to customers such as purchase history, financial status, 
product preferences, contact information, etc. A central server (COS) 77 

15 acts as a host location for a CINOS manager application (noted in text 
balloon) which is, in effect, the parent application that controls all of the 
operation and functionality of the system. 

In addition to the above-mentioned machines hosting CINOS 
routines, each PC/VDU such as PC/VDU 39, for example, has a CINOS- 

20 agent desktop interface or client application (not shown) adapted to interact 
with the parent application. Also, each machine that provides particular 
dedicated function to communication center 17 such as switch-connected 
CTI processors, IVR's, and other related equipment host instances of 
CINOS application-program interfaces (API's) to enable seamless 

25 integration of differing parameters and/or protocols that are used with 

various planned application and media types utilized within communication 
center 17. Such programs may also co-reside or be in any combination or 
hosted by themselves. Additionally, for performance purposes, additional 
dedicated network links may exist between those servers, but essentially 
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they are only performance boosters, and hence for clarity purposes, only a 
simple network is shown. 

As previously described, CINOS comprises a multi-tiered 
architecture. This unique architecture comprises an external media layer for 
interfacing with the customer or business contact, a workflow layer for 
making routing decisions, organizing automated responses, recording 
transactions, and so on, and an internal media later for interfacing and 
presenting interactions to an agent or knowledge worker. An innovative 
concept associated with CINOS involves the use of tooled process models, 
knowledge bases, and other object models as base instruction for it's 
various functions. These modular conventions may be inter-bound with 
each other, and are easily editable providing a customizable framework that 
may conform to virtually any existing business logic. 

In simple operation, and after any network level routing, COST calls 
and DNT calls including other media events arrive at communication center 
17 to telephony switch 27, and routing server 29 respectively. Network 
level routing, as defined herein, includes any intelligent implementation that 
may be in place and aided via processors 59, 61, and 22. Load balancing to 
multiple communication centers, and transferring customer data obtained at 
network-level over data-network connection 66 would be examples of such 
network-level routing. 

Once a call or other communication event registers at either switch 
27 or routing server 29, CINOS immediately identifies the media type 
associated with the call and begins it's processes depending on enterprise 
rules. For example, a live COST call may first be routed to IVR 69 whereby 
the customer can be presented with varying choices such as leaving a voice 
message, waiting in queue, receiving a call back, or perhaps an e-mail, and 
so on. Interaction by IVR 69, in this instance, will preferably be via voice 
recognition technique such as is known in the art, but may also be via touch 
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tone response or other known method. As previously described, the caller 
may elect from a number of options, such as to hold for a next available 
agent, select an automated response such as a fax back, or perhaps a later 
agent-initiated response such as an e-mail or call back. In all cases, CINOS 
5 seamlessly processes and executes the logic required to accomplish the goal 
of the caller in a media and application-independent fashion. 

DNT events are handled much the same way as described above for 
live callers. For example, an IP call may be routed to a digital equivalent of 
an IVR for interaction or queued for a next available agent, and so on. In 
10 one embodiment, IVR 69 may be adapted to handle both COST and DNT 
interaction. 

All interactions with live external media, including actual text-based 
events whether live or not, are recorded and stored in MIS 79 with an 
associated text version of the media stored as well, and becoming part of an 

15 overall threaded contact history. This is accomplished in varying ways 

according to existing parameters such as media type, whether the event is a 
live call, and so on. For example, CINOS may execute a command 
directing IVR 69 to digitally record an incoming COST call during customer 
interaction and then store the voice recording of the transaction in MIS 79. 

20 A text version of the recording either created simultaneously from the voice 
recording via voice-to-text techniques (known in the art), or created by a 
live attendant via manual annotation may be sent to and stored in DB 79. 
An IPNT call arriving at routing server 29 may be similarly recorded and 
stored in MIS 79 with an associated text version of the interaction stored in 

25 DB 79. E-mails, video calls, voice mails and so on are similarly handled. 

For example, an incoming e-mail is stored in MIS server 79 while text from 
the e-mail may be extracted and stored associated with the e-mail. 

The purpose of the text version of the event is twofold. Firstly, a 
complete text-based transaction history of communication center 17 may be 
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compiled and reserved for later access and audit. Secondly, an agent or 
knowledge worker may, in some instances, see the text version of the event 
at the same time that he receives routed notification of the event. In this 
way, an agent may begin mental preparation before taking a call. The text 
5 version of an event must be machine-readable and human readable at times 
displayed. Interactive media-independent viewers, part of the agent's client 
application, may be used to disseminate information which may initially not 
be human readable. 

It is important to note here that the text-based version of an event 

10 may or may not be a complete and verbatim rendition of an actual media 
event. For example, an e-mail may contain many documents each having 
many pages of text. Therefore, the text-based version of a particular e-mail 
event may simply contain the name and particulars regarding the author, a 
purchase order, and a list of the enclosed documents by title, and basic 

15 content or memo as well as a possible manual annotation. The attachments 
to the e-mail may be stored separately, and be also cross-indexed and 
retrievable. Seeing the purchase order when the event is routed to an agent 
desktop tells the agent that this e-mail is important. 

A fax, stored originally as a bit-mapped document, may be 

20 converted to text in the system via optical recognition (OCR) technique 
wherein sometimes only certain content such as the authors contact 
information, basic intent of the fax, and perhaps special numbers or codes 
contained in the original fax are recorded in a text version 79 , sometimes 
the whole text is OCR'd, while the original fax is stored in it's entirety in 

25 DB 79. Such codes or numbers that are specifically parsed from actual 
media may be part of a unique coding system set up by the enterprise 
whereby customers are directed to include such codes or numbers with their 
orders, service requests, and so on. 
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Parsing text messages is accomplished via a text-analyzer known to 
the inventor. In other non-text media types, such as video or graphics, 
descriptive notes may be taken via live attendant and stored in DB 79 as 
previously mentioned. Voice recognition technology may also be used in a 
5 case of recorded sound or video with sound. All transactions regardless of 
media type are thus recorded and stored according to enterprise rules with at 
least a meaningful part of the content if not all of the content of such 
transactions converted to text and stored in DB 79 associated with the 
recording of the event. Again, the importance of the text version is that the 
10 extracted knowledge of the transaction therein is in machine-operable code, 
allowing search and cross-referencing functions that may otherwise not be 
possible. 

After incoming events are analyzed and processed with regards to 
queuing, recording, storing, etc. CENTOS decides the disposition paths of 

15 each event. For example, live calls in queue are routed to live agents if 
available, if this is the priority action in the enterprise rules. E-mails are 
either routed to next available agents using a push technology, or simply 
stored in MIS server 79 where they may be retrieved by agents after 
receiving notification. Recorded events such as IVR voice requests are 

20 stored in MIS server 79 where they may be retrieved by agents, and so on. 

By the use of routing and routing notification events, any media may 
be routed to an appropriate agent based on skill, or any other rule-based 
routing method over LAN 55. Actual multimedia events may be accessed 
from MIS server 79 at the agent's discretion, or by rule, and text-based 

25 versions of those events stored in DB 79 may be mirrored and routed to the 
agent along with notification of the incoming event. 

Other services may be performed by CINOS such as responding to 
media requests without agent participation via initiating automated fax 
responses, out-bound dialing campaigns wherein recorded information is 
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given to a customer perhaps concerning an order placed by the customer, 
and so on. Networking via business or chat applications between several 
business partners, customers, agents, and so on, is possible wherein each 
entry may be stored in DB 79 as part of a discussion thread including 
5 responses of another media type, perhaps initiated by a communication- 
center agent to one of the participants during the discussion. 

As a general rule, full multimedia storage is done in a mass storage 
server, and linked by cross-indexing to the database. Depending on the 
business model, full text or only partial annotation is stored in the database, 
10 or a mix therof, e.g by media type. 

In addition to supporting a wide variety of applications and protocol, 
CINOS is provided with the tools for building media-independent self-help 
wizards that are adapted for problem solving and reduction. Similarly, 
external and internal interaction media viewers are provided and adapted to 
15 support any media of choice. 

CINOS uses object modeling and linking techniques that are known 
in the art to effect much of it's goal of presenting a seamless customer 
interaction with an enterprise agent or knowledge worker operating in a 
communication center such as center 17. For example, an interaction object 
20 model (IOM) represents a transcript of all interaction history stored in DB 
79 and provides an audit trail of the state of transactions of all interactions. 
An interaction process model (IPM) controls how events are handled within 
the operating system. 

An additional set of models handle how agents receive their routed 
• 25 media such as via traditional push model, blended push model, publish and 
subscribe model, or interrupt model. Prioritizing interaction events may 
also be accomplished through varying the push theme or scheme. For 
example, traditional push technology for e-mail means that only e-mail 
(media type) is being worked on by an agent. By blending the push model 
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with a publish and subscribe model, the interrupt model is created wherein 
the agent may subscribe to various routed media such as answering phones, 
and responding to faxes, but may be interrupted for an important interaction 
of another media type such as e-mail and so on. In this way an agent's time 
5 may be utilized according to enterprise rules within an automated 
environment. 

Outbound campaigns may be configured according to enterprise 
rules and media preference using a single rule-set knowledge-base. This 
single set of outbound tools can be used to initiate customer dialog via 

10 predictive dialing, e-mail push, automated recorded messages, and so on. 

It will be apparent to those with skill in the art that common object 
modeling (COM) can be used to create virtually any type of model for any 
type of enterprise situation. It is the intention of the inventor to provide the 
applicable control codes known in the art for building process and object 

15 models and enabling the linking and interaction between the models. As 
previously described, it is partly the fact that CENTOS uses these various 
models and knowledge bases to achieve desired interaction that sets it above 
current-art systems. The inventor knows of no such network interfacing 
operating system that is based on the above described technology. 

20 CINOS may be implemented in a number of different topologies. 

For example, CINOS may be implemented as a centralized topology with 
one communication center as shown here in Fig. 1, a distributed topology 
wherein a single communication center may span multiple physical 
locations, a segmented communication center wherein a single pool of 

25 agents services more than one company or customer base, or a wide 

communication network wherein a plurality of communication centers such 
as center 17 cooperatively service a common pool of customers or a 
customer base. Enterprises involved in commerce such as large financial 
institutions hosting many geographically separate communication centers 
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may build their entire networking system using CINOS architecture in 
standardized and distributed fashion. There is no limitation to the type of 
enterprise that may use CINOS as it may be tooled to accommodate 
virtually any network architecture linked to a communication center having 
5 DNT capability. 

It will also be apparent to one with skill in the art that CINOS 
routines according to various embodiments of the present invention may be 
included and implemented at the network level without departing from the 
spirit and scope of the present invention such as in processor 61, and IVR 

10 59 in PSTN 13, or in routing node 21 in WAN 1 1 . 

Fig. 2 is a block diagram illustrating basic layers of the network 
operating system according to an embodiment of the present invention. As 
previously described with reference to Fig. 1, CINOS comprises three basic 
operating layers. They are an external media layer 83, a workflow layer 85, 

15 and an internal media layer 87. External media layer 83 interfaces directly 
with the customers or business contacts or partners as illustrated via 
customers a and b, and business contact c. The bi-directional arrows 
beneath each of the above mentioned participants illustrate interactive 
participation with CINOS on the customer side. 

20 External media layer 83 may, in one embodiment, be a multifaceted, 

web-based self-help interface providing news information and a host of 
other services that may be personalized by the customer. In many respects, 
external media layer 83 in this embodiment is similar to a web browser. 

Workflow layer 85 comprises 3 basic function categories beginning 

25 with a content analysis category 89 wherein textual analysis, voice analysis, 
IVR interaction, recording and storing takes place. A next category is 
context resolution 91. Context resolution involves customer identification, 
business process binding, preparation for routing, and so on. A third 
category termed interaction routing 93 comprises various processes 
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associated with the presentation of the interaction to agents, service persons, 
knowledge workers, business partners, customers and the like, that is, all 
transaction partners. Category 93 covers queuing, skill-based routing, 
automated treatment, workflow models, and so on. 
5 Internal media layer 87 comprises an agent desktop interface not 

shown in Fig. 1, but described in more detail below. Both external layer 83 
and internal layer 87 contain the required tools for enabling media and 
application-independent interfacing such as previously mentioned self-help 
wizards, media viewers, and other controls as prescribed via enterprise 
10 rules. 

Internal media layer 87 provides an agent with, among other options, 
information about the customer or contact, information about current or 
historical business processes, information about current interactions and 
their relationship to business processes, and a knowledge-base to guide the 

15 agent or knowledge worker with interaction response and workflow. An 

agent a, and agent b, and a knowledge worker c are shown herein interacting 
with the system as illustrated via bi-directional arrows. The skilled artisan 
will recognize these are merely examples, and there may be many more such 
persons, and interactions in some instances may be routed to machines for 

20 response. 

It will be apparent to one with skill in the art that the multi-tiered 
architecture of CINOS such as is illustrated herein may comprise many 
more or differing steps or processes without departing from the spirit and 
scope of the present invention. 

25 Fig. 3 is a flow chart illustrating basic steps performed by the 

interaction operating system of Fig. 2 related to completing a transaction 
between a customer and an agent, wherein the transaction is initiated by the 
customer. Similar steps may be accomplished in the opposite direction for 
communications initiated by an agent, as the system is bi-directional, but the 
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present example will serve to teach the inventive aspects of the system. In 
step 95, an incoming transaction, such as a live call, an e-mail, etc. , is 
received at the appropriate CTI switch (COST) or routing server (DNT) in a 
CENTOS communication center such as center 17. In step 97, customer and 
5 media type are identified and interaction proceeds. 

All transactions, whether live calls, such as video calls, DNT calls 
and COST calls, or text-based documents, such as e-mails, are recorded and 
stored in one or more mass storage devices handled by one or more database 
applications. This may be taken as server 79 of Fig. 1 , although the 

10 diagram of Fig. 1 is exemplary. 

A principle object of the invention is to extract maximum 
information from every transaction for building a knowledge base that can 
be used for dynamic management and future analysis and development. 
This is done primarily by data mining, which is applicable to machine- 

15 operable code, that is text. Because of the nature of the extraction, there is a 
difference in the way live calls and text-based media is handled. 

Discrimination as to the text nature of the media is made at step 99. 
If the media chosen by the customer is already text-based, then the 
transaction is recorded as received (101), and a data mining application 

20 extracts important information in step 103 and stores it in the knowledge 
base. The distinct portions and versions of the transaction, such as the 
originally recorded version and any extracted data are related to one another 
and to other knowledge previously stored, and become part of a threaded 
interaction history associated with an ongoing interaction and ultimately of 

25 an overall contact history. 

If the media chosen by the customer is determined in step 99 to be a 
live interaction such as a COST or IPNT call, then the existing knowledge 
base is accessed at step 107, and the call is routed to the best fit agent. This 
may, of course, be done in a number of ways, such as an ADC, skill-based 
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routing as known to the inventors, transfer to an IVR for automatic 
processing, and so on, as may be dictated by enterprise rules. If routing is to 
an agent, customer information may be retrieved from CIS server 57 (Fig. 1) 
and sent to the agent's PC, and appropriate scripts may be provided to guide 
5 an agent in interacting with the caller. 

In step 109 the actual transaction is recorded as it takes place, which, 
in the case of live calls, may be a video or an audio recording or a 
combination of both. Preferably the recording is digitized. 

In step 1 1 1, a maximal text version is prepared from the actual 

10 transaction. The ability to do so depends to a degree on the sophistication 
of the system. This process may be as simple as a person adding notes for 
annotation or as sophisticated as a voice-to-text application preparing a full 
text version as the transaction transpires. 

In step 1 13 the text version is mined for data and resulting 

15 knowledge is stored in the appropriate knowledge base for future use, and 
added to overall record with appropriate cross-referencing. 

It will be apparent to one with skill in the art that there will be many 
routines comprising various steps for performing different processes as may 
be determined by enterprise rules which may likewise vary depending on, 

20 among other considerations, company type, product and or service type, 

communication center architecture, whether or not the system architecture is 
centralized or distributed, and so on. The embodiment taught herein is 
meant only as a basic example of process functionality related to CENTOS 
processing of an incoming event. 

25 Fig. 4 is a block diagram illustrating agent-desktop function 

according to an embodiment of the present invention. An agent-desktop 
client 1 15, part of the CINOS overall architecture, enables an agent or 
knowledge worker to configure and control his or her interface to the rest of 
the system and to external media. Client 1 15 may be personalized 
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according to a particular agents parameters. A desktop interface 1 17 may 
appear and function much like a personalized web-browser containing many 
similar attributes related to network capabilities including full multimedia 
function, software tool kits, linking and embedding capability, and so on. 
5 An HTML client application 119 oversees all of the network 

capability previously mentioned. In this embodiment for example, HTML 
client 119 communicates with an Internet information server 121 using 
HTTP protocol which is standard. Client 1 19, if provided minimally, may 
be used in conjunction with an Internet browser for full multimedia 

10 function. In some embodiments, it may be maximally provided to be a fully 
featured client with full web browser function. For example, an agent may 
create and edit web forms, web pages, embed controls into such web-based 
forms or pages to provide certain customer interaction mechanisms in 
addition to having a fully functional navigation tool at his disposal. 

15 In another embodiment, Server 121 may be a server on a private 

network or corporate WAN instead of an Internet server. In a preferred 
embodiment, however, any number of servers on the Internet and/or linked 
to a WAN other than the Internet may communicate with client 1 19 as it 
intended to support all existing and known communication protocols. 

20 A windows client 123 is provided to seamlessly integrate existing 

applications on the agent's PC to network applications and processes. This 
may be implemented via a desktop tool-kit 125 that contains all of the 
required controls for building, integrating and customizing the interface. 
A business-logic layer comprises business object models 129, 

25 hereinafter termed business objects 129, representing contacts, interactions, 
knowledge-bases, events, routing processes, and other system routines. 
Integration and interaction of the various described desktop components 
with these logics is accomplished via common object modeling (COM) 
which is known in the art and available to the inventor. Desktop to CTI 
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integration is accomplished via controls provided or created with a CTI set 
of tools or tool kit (not shown). For example, if the enterprise desires to 
blend voice and e-mail, the CTI tool kit would be used to build and integrate 
the interface. 

5 Existing network applications such as CIS, enterprise resource 

planning (ERP), Commerce, and the like interact with various business 
objects using COM and may also interact with a physical database using 
ODBC and SQL. 

10 

Customer Interface Media Window 

According to a preferred embodiment of the present invention, 
CINOS access by customers of an enhanced multimedia communication 

15 center, such as center 17 of Fig. 1, is controlled by means of a customer- 
facing media interface, by which customers may be identified and even 
categorized according to numerous criteria. In some cases access may be 
controlled through subscription, or according to other qualifying criteria 
such as may be deemed appropriate by the enterprise. For example, if the 

20 enterprise is an exclusive investment club, membership may be required. 

Categorizing criteria may include demographic information such as income 
level, credit history, or any other attribute that may be quantified and used to 
categorize a customer. 

An enterprise-controlled access point may be defined as an 

25 interfacing window or portal created and maintained at a typical customer 
entry point in a network as may be known in the art. Such interfaces may 
take the form of a WEB-based customer interface (a WEB page), an 
interactive voice response (IVR) unit, a service control point (SCP), or some 
other customer-facing system or apparatus as may be known in the art. 



-27- 



For the purposes of this specification, an example of an enterprise- 
controlled WEB-form access and interface window is illustrated as an 
example for a preferred embodiment. The inventor deems such an interface 
to be most adept in offering best-fit media options while remaining 
5 accessible to a large customer or client base. 

Fig. 5 is a block diagram of a WEB-form customer interface 
according to an embodiment of the present invention. WEB form 133, 
hereinafter termed access window 133 ? is provided to be a part of an 
enterprise's WEB page which may be accessed through Internet connection 
10 and navigation, as is known in the art. Widow 133 is part of the CINOS 
software architecture described above, and represents the initiation of any 
customer interaction with the hosting enterprise. A WEB counter 143 is 
provided and records the number of visits to window 133. 

Window 133 is built and edited using COM codes available to the 
15 inventor and typically found in tool kits adapted for the purpose of creating 
interactive displays on a WEB page. Such a tool kit may be located on an 
agent's desktop, perhaps part of an agent's HTML client such as client 119 
of Fig. 4. In one embodiment, it may be part of a system administrator's 
tool kit. 

20 Window 133 contains interactive options directed at various 

categories and functions. For example, a new client section 135 contains 
interactive options related to adding a new client to the active customer base 
of the enterprise. A customer service section 137 contains interactive 
options presented to existing clients needing service. A new order section 

25 139 contains interactive options presented to existing clients wishing to do 
new business. 

Each offered interactive option is an embedded executable function 
having the appropriate links to other system areas of CINOS such as may be 
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relevant to the immediate interaction such as to services offered, routing 
routines, database searching, interaction recording, and so on. 

An innovative function of window 133 is to provide front-end 
control of access to the enterprise by existing and potential 
5 clients/customers. For example, as a client, contact, or potential client 
interacts with the various media and functional options presented by the 
enterprise in window 133, he or she is being directed according to enterprise 
rules in such a way that he or she may first be qualified or not to patronize 
the enterprise. Secondly, the contacting person may be categorized and 

10 sorted as to type of qualified customer. Thirdly, the person contacting 

person may be directed to pre-selected media options by the enterprise for 
various services offered including but not limited to routing live 
interactions, and so on. 

In a preferred embodiment of the present invention, access window 

15 133 is fully customizable, based on customer data and enterprise rules with 
the focus of such customization directed toward benefiting the enterprise 
and ultimately the client. That is, the client's options within window 133 
are pre-selected and preferred by the hosting enterprise based in part on data 
about the client, details about the client's communication equipment and 

20 software, and enterprise rules and constraints. In some embodiments, the 

client may aid in customizing window 133. However, as it is desired by the 
enterprise to provide service in a cost-effective manner, the client will be 
presented with options as preferred by the enterprise in most cases. 

To further illustrate, refer now to new client section 135. If window 

25 1 33 is part of the enterprise WEB page, as is the case with this example, 

there will be a variety of visitors which may or may not be pre-qualified by 
the enterprise. Therefore, an interested party would begin (and be restricted 
to) taking a new client survey, illustrated as one of the options in section 
135. If the enterprise rules require this as a first step, then the other options 
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may be enabled only upon completion of the survey. By choosing new 
client survey, a second window may contain various survey options such as 
via e-mail, interactive voice recording, type and send method, or the like. 

Information taken in the client survey is recorded and entered into a 
CENOS database such as DB 75 of Fig. 1. Such information may also be 
compared against enterprise rules or constraints, and other known 
information as may be available to the enterprise. Assuming the client is 
now recognized by the enterprise, the client's media hardware and telephony 
information may be recorded for future interaction purposes. Such 
information may include the client's personal computer parameters 
including modem type, Internet connection type, computer platform type, 
type of Internet phone application installed, etc. Similarly, COST telephone 
parameters may be recorded, such as personal phone number, business 
phone number, cellular phone number, forwarding numbers, and so on. 
Such data will influence latter customization of his personal window 133 
for the particular client including the types of media that will be offered. 

Finally, the client may be asked to create a password for the purpose 
of accessing CINOS. A section 141 is provided containing a network log-in 
option along with download sections for obtaining permanent and or 
temporary software as may be desired or needed, or, in some cases, required 
for the client to access certain services, view certain content, and so on. 

Section 137 presents media options for clients seeking customer 
service from the enterprise. These options are, in a preferred embodiment, 
presented in a customized or personalized fashion within the client's 
window 133 as was described above. Therefore, each client patronizing the 
enterprise may access a version of window 133 that differs in look and 
functionality than that of another client. In this example, service section 
137 contains options for e-mail, chat program, fax program, a self-help 
wizard, and a voice wizard. Other media types may be added or subtracted 
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from the client's window 133 depending on any of several criteria. 
Personalization of widow 133 takes into account client information as 
stored in CDSfOS database 75, service-agent media availability and 
preferences, and perhaps any overriding enterprise rules. Unless and until a 
5 client is identified there are typically no options presented to the client for 
continuing a transaction with the enterprise. 

For an identified client, by selecting the e-mail option, the client's 
preferred e-mail program may be activated for the purpose of sending a 
message to or soliciting a reply from a service agent. By selecting chat 

10 program, the client may be launched into a scheduled service seminar 

featuring many clients interacting with a service expert regarding a certain 
subject. One enterprise rule regarding section 137 may be that there is no 
telephone or I-phone media option for customer service for a client in the 
absence of an ongoing project with the particular customer. In this sense an 

15 ongoing project includes any unfinished business that the client is involved 
in with the enterprise. 

Self-help wizards and voice wizards as illustrated in section 137 
may be offered to help a client resolve an issue without taxing further 
resource. Such wizards may be customized based on a client's recorded 

20 data, perhaps confirming past interactions, providing account or order 

status, and so on. In some embodiments, selecting an option might avail 
several additional options. For example, selecting chat program may avail 
three possible chat programs to choose from with different schedules, 
content, and functionality attributed to each individual program. 

25 New order section 139 in this example contains various options 

adapted to facilitate placing orders. The options as illustrated herein 
include, but are not limited to, I-phone, call back, promotional models, 
video presentations, an on-line viewer, and an order wizard. Interaction is 
the same as was stated with regards to section 137. For example, selecting 
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promotional models, accesses a database containing the current promotional 
information and features of products which may be viewed interactively by 
the client using an on-line viewer offered as one of the functional options 
(tool). The options presented in the New Orders section may also be 
5 customized according to client identity, demographics, transaction history, 
and enterprise rules. 

On-line viewers may enable the client to view documents that are 
not supported on his computer platform. Selecting video presentation may 
avail several types of videos for viewing ,such that the client may choose 

10 one. If the client does not have a viewer installed on his computer which 
will support the offered video, perhaps the on-line viewer may play the 
video, or the client could download a temporary viewer from section 141, 
etc. Selecting call back may bring up a second array of media choices made 
available by the enterprise for receiving a reply interaction from an agent. 

15 By providing a controlled interface window such as window 133 the 

enterprise may control routing and interaction right from the beginning of 
customer contact. Through the innovative method of linking and reporting 
to other CINOS functions, and repositories, much real-time personaliation 
of window 133 according to enterprise rules and customer parameters may 

20 be made automatically. For example, if a client's history indicates a 

propensity toward frequent buying, an I-phone option may be presented in 
customer service section 137 in his window 133 immediately after such a 
determination so that he may get direct customer service at all times. 

Certain media options, as described above, may be afforded a certain 

25 priority over one another regarding interaction with the enterprise. For 
example, a VIP client may have live interactive media choices offered in 
window 133 such as I-phone, call back to COST phone, video phone, etc. A 
client known for infrequent contact or troublesome interactive history may 
be limited to text-based interaction such as e-mail and so on. 
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As an integral part of CINOS functionality, window 133 acts as a 
portal through which existing and potential clients may be screened, 
categorized and routed according to enterprise rules. Customer interfaces 
such as window 133 may be provided at various locations on a WAN such 
5 as the Internet without departing from the spirit and scope of the present 

invention. Such portals may exist in different geographic regions, and may 
be created for differing customer bases such as one for Latin America, and 
one for the pacific rim, and so on. Instances of CINOS routine may be 
distributed widely over a network. 

10 Although the example provided herein is of a WEB form, it will be 

apparent to one with skill in the art that a CTI counterpart may be created 
for the COST telephony network. Such a case may be a CINOS enhanced 
IVR at an SCP or customer access point in the COST network. 

CINOS, as previously described, optimizes customer/agent 

15 interaction in a manner which is economical and cost efficient to both the 
enterprise and the patronizing client. The customer interfacing window as 
taught herein with regards to Fig. 5 is innovative in that it is a fully 
customizable portal that facilities seamless integration between clients and 
enterprise agents according to enterprise rules. Further innovation is 

20 evident in that client data is fully and seamlessly integrated with CINOS 

intelligence and enterprise rules regarding routing of interactions and other 
constraints or limitations that are programmed into the system. In effect, 
logic from the front end, or customer side, to the back end or agent side is 
linked and accessible to all appropriate CINOS routines which include 

25 applicable CTI CINOS routines. The various customer interfacing logic is 
are explained more fully below in a series of process logic steps in a flow 
chart. 

Fig. 6 is a flow chart illustrating media-presentation and customer- 
interface logic steps according to an embodiment of the present invention. 
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In step 145, a visitor registers at an enterprise's WEB page. The visitor is 
identified according to enterprise rules in step 147. In step 148 CINOS 
determines the current status of the visitor after searching known client and 
contact data records. For example, the visitor may be a potential new client, 
5 an existing client, or an existing business contact. Although not specifically 
illustrated, a potential or new-business-client is not typically logically 
separated from a potential new-client until further process ensues in step 
149 with regards to qualification via survey. 

If the visitor wishes to be a client, he may log-in to the network 

10 system in step 159. Log-in may be automatic in the event that CINOS 

remembers the client's assigned password, or perhaps typing the password 
or other code may still be required for security reasons. At the time of log- 
in, window 133 is presented in personalized fashion according to client data 
and enterprise rules in step 161. In step 163, interaction between an 

15 enterprise entity and the client begins with a media type that is offered by 
the enterprise and selected by the client. An enterprise entity, as 
immediately described above, is herein defined as an agent, knowledge 
worker, service person, or any other live attendant, as well as any entity 
constituting an automated response action such as an automated fax, an 

20 IVR, automated file downloads, etc. 

At step 148, if it is determined that the visitor is new, then a new 
client survey is conducted in step 149. Step 149 will determine if the new 
visitor is a client or business contact via the survey process. As described 
with reference to Fig. 6, the client survey may be conducted using a variety 

25 of known techniques and media. Presuming that a new visitor qualifies as a 
client or business contact in step 149, he or she may be asked to create a 
password in order to provide access to CINOS. In step 153, the client's 
appropriate communication and system parameters are recorded for future 
reference and for use in customizing window 133. 
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At step 155, a client instance of CENTOS, or perhaps another enabling 
application, may be presented for download by the client. In some 
embodiments, there may be no required software for download. Therefore, 
step 155 may be considered optional in this regard. In step 157, the new 
5 client may log-in to the network system and begin interaction. Because the 
client, in this case, is accessing the system for the first time, the steps 
wherein he would obtain a customized window and begin interaction with 
an enterprise entity are not shown as intermediate configuration of media 
choices, product preferences, and the like, may still be required before a 

10 customized interface may be presented. In one embodiment, the client may 
not see a customized window until the next time he or she attempts to 
access the network. 

Steps 165, 167, and 169 for an existing business contact as 
determined in step 148 are similar to steps 159, 161, and 163 for an existing 

15 client although the rules for interaction such as media used, personnel 

involved and so forth will be different. For example, in step 167 an existing 
business contact may be offered the option of using a network-collaboration 
program wherein I-phone, file sharing, video conferencing and the like are 
inherent to that one application. 

20 It will be appreciated that there are many possible logic sequences or 

steps that may be followed in interfacing and enabling interaction between a 
client and an enterprise entity without departing from the spirit and scope of 
the present invention. Fig. 6 presents just one possible example of many. 

It will be apparent to one with skill in the art that the rules governing 

25 the types of media offered to clients may be based on a combination of 

variables such as may be decided upon by the enterprise without departing 
from the spirit and scope of the present invention. Likewise, offered media 
types may be added or withheld from a client over a period of time based on 
such variables. Moreover, such additions or subtractions of media 
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availability with regards to customer interface window 133 may be 
automated and based on calculated variables. 

In one embodiment, a client may add or subtract media choices if 
desired, however, the enterprise may reserve the right not to engage such 
5 media if added by a client. 

In one embodiment, special application-independent media viewers 
such as the viewer offered in section 139 of window 133 of Fig. 5, are 
offered to clients and possessed by agents so that initial illegible 
information may be made human readable regardless of the authoring 
10 application used by the agent or the client in the process of interaction. 

Rules-Based Storage and Threading of Multimedia Interactions 

In a preferred embodiment of the present invention, all CINOS 
15 controlled interactions with customers or business contacts are recorded and 
stored in a contact history comprising a MIS database and a text database 
such as were described with reference to copending application P3313PA, 
and described above. That is, actual multimedia interactions are recorded to 
one database or to a section of one database supporting all multimedia types 
20 used in the communication center, and text-based versions are stored in 

another database or portion of the same database. All of the actual recorded 
transactions and text versions are related as a threaded contact history which 
may be separate from or part of the same database as will be further 
explained below. 

25 Fig. 7 is an exemplary overview of a multimedia-interaction storage 

system within a communication center according to an embodiment of the 
present invention. A system architecture 171 is illustrated solely for the 
purpose of providing just one of many possible architectures in which the 
methods and apparatus of the invention may be practiced. Architecture 171, 
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which in a preferred embodiment comprises both conventional and DNT 
apparatus, is exemplary of an architecture that could facilitate CINOS 
according to an embodiment of the present invention such as is also the case 
of Fig. 1 

5 At the heart of the storage system is a mass-storage repository 187 

adapted to store multimedia interactions as well as text-based related files. 
Repository 187 may utilize any form of digital storage technology known in 
the art such as Raid-Array, Optical Storage, and so on. The storage capacity 
of repository 187 will depend directly on it's implementation with regard to 

10 the size of the communication center and predicted amount of data that will 
be stored and kept by the system. 

In this example, repository 187 is divided logically into two sections. 
One section, multimedia information system (MIS) 189, is responsible for 
housing all multimedia interactions defined as media that is not text-based 

15 such as audio, video, and graphics-based media. All multimedia 

interactions are stored in MIS 189 whether incoming, outgoing, or internal. 
A second section, herein referred to as text section 191 is responsible for all 
text-based interactions as well as text versions related to non-text files. 
Sections 191 and 189 of repository 187 are analogous to MIS 79 and DB 75 

20 of Fig. 3. 

Repository 187 is connected to a communication-center local area 
network (LAN) 195. Repository 187 is accessible via LAN 195 to 
authorized personnel within a communication center such as agents, 
knowledge workers, or the like, and may, in some instances, also be made 

25 available to clients communicating with the call center. A network router 

(RTN) 175 is shown connected to LAN 195 via network connection 203. In 
this example, network router 175 is the first point within a communication 
center wherein DNT media arrives. Network router 175 is exemplary of 
many types of routers that may be used to route data over LAN 195. An 
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Internet-protocol-network-telephony (IPNT) switch 176 is connected to 
network router 175 via data link as is known in the art. IPNT switch 176 
further routes or distributes live IPNT calls that do not require routing to a 
live agent. IPNT calls that are routed to live agents are sent over connection 
5 203 to LAN 195 where they reach agent PC/VDU's (not shown) or DNT- 
capable phones (not shown) as illustrated via directional arrows. 

An object of the present invention is to record all multimedia 
interactions and store them in MIS 189. A further object of the present 
invention is to similarly record text versions of and text files related to all 

10 multimedia interactions and to store them in text-based section 191. For the 
purpose of the present invention, a text version of a non-text file is defined 
as a sufficient text rendition or description of a corresponding multimedia 
interaction. Still another object of the present invention is to provide an 
innovative mechanism wherein authorized persons may access any 

15 particular block of text and if desired, call up the actual media to which the 
text relates. 

More detail regarding the order and manipulation of repository 187 is 
described further below. 

Creating text-based versions of live multimedia interactions may, in 

20 some cases, be accomplished via an automated method. For example, a 
digital voice attendant 197 is provided and linked to IPNT switch 176. 
Digital voice attendant 197 may be of the form of a DNT-capable IVR or 
other digital voice-response mechanism as may be known in the art. Such 
automated attendants may interact with a voice caller instead of requiring a 

25 live agent. A speech to text converter 199 is provided and linked to voice 
attendant 197. As digital voice attendant 197 interacts with a caller, speech 
to text converter 199 uses voice recognition technology to convert the audio 
speech to text. Such text may then be stored automatically into text section 
191 and related to the also-recorded audio data. 
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It will be apparent to one with skill in the art that as speech 
recognition technologies are further improved over their current state, which 
is adequate for many implementations, reliable text versions of audio 
transactions are not only possible but practical. Such speech to text 
5 conversions are used here only for the convenience of automation wherein 
no live attendant is needed to transcribe such audio data. The inventor is 
familiar with such converters as used in the CINOS system according to a 
preferred embodiment. Such converters provide convenience in the practice 
of the present invention but are not specifically required to achieve the 

10 objects of the present invention. 

Manual transcription may also be used to convert audio/video to text 
or code that may then be entered into text section 191. For example, a live 
attendant 201 is shown connected to LAN 195. Attendant 201, in this case, 
may be given the responsibility of transcribing audio files from speech to 

15 text and annotating video or graphics files for the purpose of creating text 
files related to the non-text data. One or more live attendants such as 
attendant 201 may be provided for this purpose. Some media arriving at a 
communication center such as the one represented via architecture 171 will 
already be text-based and therefore require no conversion or annotation. 

20 Short e-mails, Faxes, word documents, and so on are part of this media 
category. 

An automated services system 193 is illustrated as having a direct 
connection to section 191 of the data repository. System 193 is provided for 
certain text-based interactions, as described above, wherein a complete text 
25 record of the interaction may be mirrored, or otherwise created and stored 
into text section 191. Such automated services may include but are not 
limited to automated e-mail and fax systems. For example, a fax may be 
sent and mirrored into section 191 or, perhaps recreated using an optical 
character recognition (OCR) technique and then entered. Physical text- 
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documents such as legal papers and the like, may be automatically scanned 
into text section 191 before they are sent to clients. There are many possible 
automated techniques for entering text files into a database. Such methods 
described with regards to automated services 193 are a convenience in 
5 practicing the present invention but are not specifically required to achieve 
the objects of the present invention. 

With respect to the dual capability (COST7DNT) of architecture 171, 
a central telephony switch 173 is provided to be a first destination for COST 
calls arriving from, for example, a PSTN network. Switch 173 may be a 

10 PBX, ACD, or another known type of telephony switch. Internal COST 

wiring 182 connects telephony switch 173 to agent's individual telephones 
(not shown). Switch 173 is enhanced via a processor 179 running an 
instance of T-server and an instance of Stat-server, which are software 
enhancements known to the inventor and have been previously described. 

15 Such enhancements provide CTI applications, such as intelligent routing, 

but are not specifically required to practice the present invention. CINOS as 
previously described is adapted to be integrated with such software when 
present in a CINOS enhanced communication-center. 

An intelligent peripheral in the form of a COST IVR 177 is provided 

20 for the purpose of interacting with callers seeking information and the like 
who do not require connection to a live agent. IVR technology may 
comprise voice response, touch tone interaction, or a combination of these 
technologies. IVR 177 is linked to processor 179 and also to automated 
services 193. An example of an IVR interaction may be the presentation to 

25 a caller of options for using an automated service such as those described 
above, or perhaps waiting for a live agent. 

A CTI to DNT interface 181 is provided for the purpose of 
converting COST interactions to digital mode compatible with DNT so as to 
be adapted for digital storage and interaction according to CINOS 
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functionality and enterprise business rules as described above. Interface 181 
is not specifically required to practice the present invention so long as 
appropriate application programming interfaces (API's) are provided for 
equipment that interfaces with CINOS. Bi-directional arrows illustrated 

5 between interface 181 and IVR 177 represent the ability to route interactions 
in either direction. COST to DNT conversion may be accomplished in IVR 
177 in addition to or in place of interface 181. The connection architecture 
presented herein is exemplary only. 

A speech to text converter 185 is provided for converting audio from 

10 the CTI side to text for entering into text section 191 as was taught with 
regards to converter 199 on the DNT side. Actual recorded media 
interactions are illustrated entering MIS 189 after text versions are rendered 
and entered into section 191, however, this is not required. In some 
instances text versions of multimedia interactions may be rendered after the 

15 interaction is stored. There is no limitation regarding sequence. It is 
sufficient to say that converters 185 and 199 are capable of real-time 
conversion and entry. 

Server 183 shown connected to LAN 195 is adapted to host a 
CINOS MGR.(operating system) application which provides control and 

20 organization with regard to various functions provided by the CINOS 

system as a whole. The storage architecture represented herein by element 
171, and all it encompasses in this embodiment, is meant only to be an 
example architecture as may be dedicated to the storage and organization of 
communication-center data according to enterprise rules. 

25 A unique method termed multimedia threading by the inventor is 

used in a preferred embodiment of the present invention for relating each 
multimedia interaction whether incoming to, out going from, or internal to 
the system, such as between an agent and a supervisor. This innovative 
process allows agents or other authorized personnel to access text data and 
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ability cross-reference the data to actual recorded multimedia interactions 
which may be displayed and played back. 

Fig. 8 is an illustration of a relational diagram as might be displayed 
on a display monitor, representing entities stored in the databases described. 

5 The blocks of Fig. 8 illustrate threaded text-blocks and their relationship to 
stored multimedia according to an embodiment of the present invention. 
Repository 187 comprises section 191 and 189 as illustrated in Fig. 7. 
Section 191 contains text versions of interactions that are related by such as 
chronology, issue, participants, company affiliation, and the like. The text 

10 documents and versions of non-text files, represented in this case by icons, 
are shown related by serial position. For the sake of clarity regarding the 
innovative threading according to an embodiment of the present invention, a 
brief description of prior art threading follows. 

Threaded dialog as is known in prior art involves a system of strings 

15 or threads that are identified as being inherent to a single entity or subject 
matter wherein the dialog (questions and replies) is about that subject or 
about a question or subject that an entity has brought forth. A threaded 
dialog may be finite dialog (is closed at some point) or it may be ongoing. 
Typically, a thread, which connects or associates the pieces of dialog, 

20 contains all of the dialog in the order that it happened such as in 

chronological order. Threading may be implemented based on other criteria 
as may be appropriate for a particular situation or by particular rules. 

As previously described with reference to the background section, 
prior art threading techniques are confined to text such as with an on-line 

25 message board or the like. The inventor knows of no system that supports 
full multimedia interaction. The innovative implementation taught below 
integrates the text-based thread with stored multimedia interactions such 
that one may interact with the thread and access various stored media 
associated with the thread. 
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Referring again to Fig. 8, a customer 205 is illustrated as having two 
threads. They are issue I and issue II. Customer 205 has an assigned 
number XX-XX that identifies him or her with respect to the CINOS 
system. Issues I and II may comprise sales dialog, purchasing dialog, or any 
5 other type or purposed dialog as may be generic to the hosting enterprise. 
Customer 205 may well be a business contact, or even an internal agent 
practicing dialog with a supervisor or the like. 

A series of icons 225-239 represent the type of media stored for each 
text block (text not shown). For example, issue I comprises first an e-mail 
10 text followed by a fax text, WEB text, and V-phone text. In this case, a 
time stamp or other known method may be used to insure that each text 
block is in order. Icons 225-239 are interactive pointers or links to the 
actual media interactions that they represent. That is, the first block of e- 
mail text is associated with an interactive icon, in this case icon 225. By 
15 clicking on the icon a with a pointer device, the actual e-mail may be 
accessed and viewed. In an alternative aspect, not only the actual 
transaction may be presented to a user for review, but related files may also 
be listed or otherwise presented for selection and review. 

A logical link 207 represents cross-referencing capability between 
20 sections 191 and 189. Dialog may be threaded according to a wide variety 
of business rules. For example, a thread or string may represent dialog 
about a customer, product, agent, group of agents, group of customers, and 
so on. An identifier is assigned to an entity and to all the communication 
events to or from that entity, or those in which the entity may have been 
25 involved such as a group discussion or chat. In this way all interactions may 
be organized and stored accordingly. 

In one embodiment, keyboard commands could be used to cross- 
reference to actual media instead of icons. In another embodiment, text 
versions of actual media are fully viewable with the text itself appearing in 
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interactive form whereupon a double click may call up the associated media 
and so on. There are many variations within the scope of the invention. 

Although actual recorded media interactions are, in this 
embodiment, stored in MIS 189, there does not have to be two separate 

5 databases (one for text and one for actual media). All data may reside in 
one database and be sectioned in storage. For example, one click on the 
customer name may bring up text only, while a double click on the text 
brings up the associated media. 

In MIS 189, recorded multimedia interactions are represented by 

10 icons 241-247 and 251. For example element 241 represents all recorded 

Video phone interactions. Element 243 represents all e-mails. Element 245 
represents all recorded COST interactions. Similar associations are made 
with respect to elements 247 and 251 which represent WEB interactions and 
Video mails respectively. WEB interactions 247 may include on-line 

15 orders, requests, information forms, signed certificates, and so on. 

Element 249 represents additional stored multimedia files dedicated 
to, for example, promoting the enterprises products or services. 
Promotional files 249 may comprise files of the form of any enterprise 
supported multimedia. These files may be tools that can be sent to clients 

20 upon request or perhaps periodically. 

Referring again to section 191, element 227 located on the thread 
labeled issue I represents text from a fax. Because a fax is text-based and 
not a multimedia interaction, there is no corresponding media event 
associated with it. However, the fax is threaded into the dialog according 

25 to, in this case, chronological order. A short example of a proposed dialog 
concerning issue I follows. 

Element 225 represents an e-mail sent by customer 205 to the 
enterprise requesting pricing information. An enterprise agent responds 
with a fax 227 to customer 205 containing the requested information. 
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Customer 205 then places an on-line order (element 229) along with a 
request for confirmation via video phone (element 231). Issue I may be 
closed at this point. Issue II may represent a threaded dialog concerning 
company service with regards to the customer order of issue I, or perhaps an 

5 agent-to-manufacturer dialog regarding how the order was handled with 
respect to issue I. 

In accordance with CINOS functionality as previously taught in 
descriptions above, data may be mined from repository 187 for the purpose 
of enhancing service to customer 205. Mined data may be used to affect 

10 routing of interactions, product promotions or advertisements that may be 
sent to customer 205. In some cases, mined data may effect new dialog 
with a customer or business contact resulting in new thread additions. A 
complete contact history with interactive linking to actual recorded media 
enables the enterprise to resolve disputes more easily, better service the 

15 customer, and enhance profitability for the enterprise. 

Fig. 9 is a process flow chart illustrating logical steps taken when 
building a threaded multimedia contact-history of communication-center 
interactions according to an embodiment of the present invention. Logical 
process steps as illustrated herein are meant to represent just one of many 

20 sequences which may be implemented when building a multimedia-threaded 
contact-history. Actual steps will depend on enterprise rules. In step 209, a 
current interaction to be recorded is identified. Identifiers may include 
special passwords or codes for identifying the contacts involved with the 
interaction. The media type of the interaction is identified in step 211. If 

25 the media type is already text-based, as confirmed in step 213, then the 

interaction is prepared for entry into a database such as section 191 of Fig. 
8. Preparation may include such automated processes as scanning, 
mirroring, file conversions, and so on. Manual annotation via live 
attendants such as attendant 201 of Fig. 7 may also be performed. In step 
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223, the text interaction is entered into section 191 of repository 187 and 
takes it's place along the associated dialog thread according to enterprise 
rules. 

If the interaction is of the form of non-text media as identified in 
5 step 211, then the MIS section of repository 187, or section 189, is notified 
to accept the input. At step 219, the non-text interaction is recorded into 
section 189 of repository 187. This may occur in real time as the interaction 
takes place, or some point after the media interaction was recorded. 

In step 221, a text version of the recorded media or a text-based 

10 document related to the transaction is rendered for storage into section 191 
as part of the thread. In some instances, as described with reference to Fig. 
7, step 221 is automated via speech to text converters and occurs at the same 
time or before the recorded multimedia interaction is entered into section 
189. In other instances, text versions of multimedia interactions may be 

15 rendered after the recorded interaction is stored. A live attendant such as 
attendant 201 of Fig. 7 may be assigned to parse video and or audio for 
applicable text. Such parsed text is entered into section 191 and takes it's 
place along the thread as was described above. 

In all cases, an identifying medium is used to assign portions of an 

20 ongoing dialog to the proper location along a thread as well as provide 
identification to actual recorded media for cross-referencing such as may 
occur during a system audit or contact review. Further, the appropriate 
icons and or links are created and associated to entered text wherein actual 
multimedia may be cross-referenced in interactive fashion. Hence, the type 

25 of media may be readily identified by an auditing or reviewing agent simply 
by browsing the threaded text with accessibility to the recorded events made 
by interactive method such as clicking an icon with a pointer device as was 
previously described. As an additional benefit all of the threaded dialog, 
whether text based or not, is rendered in a form that data mining may be 
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used to create many useful relationships and to derive much useful 
information from the stored data. 

It will be apparent to one with skill in the art that the order and 
specific function of logical steps as taught herein may vary according to the 
5 type of enterprise, existing enterprise rules, and so on. For example, instead 
of threaded dialogs being inherent to a specific customer with the dialog 
being about the customers interactions, it may be specific to a particular 
agent with the dialog about the agents activities. Such differences in thread 
assignment may be incorporated into one rules-based repository. 

0 

Interactive Multimedia Viewers and Applications 



In a preferred embodiment of the present invention, CINOS users 
comprising such as customers, agents, and business associates are provided 

15 with innovative multimedia applications that are containers for dedicated 
multimedia viewers enabling a particular user to perform a dedicated 
function or functions including gaining access to and viewing media from 
selected areas of CINOS data storage. Provision of such applications allows 
any objective to be gained regarding virtually any aspect of the enterprise. 

20 These interactive applications are built from a parent application or 

container that may contain all of the interactive modules that may be desired 
to effect a specific application to be presented to a user having a need for 
such an application. 

According to various embodiments of the present invention, which 

25 are described below, the multimedia applications may be adapted for such 
tasks as placing orders, previewing products, determining customer 
profitability, calculating sales volumes, reviewing agent performances, or 
any other enterprise-conceived objective. The abilities and constraints 
applied to these unique applications are limited only by the imagination, and 
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tools available to an authorized programmer or worker, such as a knowledge 
worker, who creates the applications. 

Fig. 10 is a block diagram illustrating an interactive multimedia 
application (MA) tool kit and a diagram of a created IMA application 

5 according to an embodiment of the present invention. An IMA tool kit 225 
is provided to an authorized programmer, which may be a knowledge 
worker, for the purpose of creating special multimedia applications such as 
an IMA 239 (illustrated within tool kit) for users of CINOS, wherein users 
may access and interact with certain pre-selected data for the purpose of 

10 reaching decisions and performing certain dedicated objectives as may be 
defined by enterprise rules. IMA tool kit 225 contains executable codes or 
modules represented as building blocks by the inventors. These modules 
may be used by themselves for certain functions, or may be linked to each 
other to provide additional function in a programmed order. A good 

15 example of such a module would be a combination of COM codes used to 
build an interactive graphical interface (module), and the like. 

Among these functional modules are interactive media viewers 
(IMV's) 227 which are provided and adapted for viewing certain media 
supported by the enterprise hosting a communication center employing 

20 CINOS. Supported media types may include but are not limited to 

telephony (traditional or IP), interactive voice response (IVR), e-mails, 
WEB embedded interfaces or forms, faxes, chat programs, multiparty 
threaded discussions, etc. IMV's 227 are unique in the fact that they are 
dedicated viewers including an interactive layer that enables a programmer 

25 to impose selective control over access to multimedia files in a data 

repository by the programmed MV. For example, CINOS users may be 
assigned an identification code or number which will also be tagged to all of 
their stored interactions as described elsewhere above with reference to Fig. 
9. These codes may be used to associate individuals with limitations and 
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constraints from viewing media that is not part of their own contact history 
(for example). Other limitations or constraints may also be applied to 
MV's 227 as may be conceived and implemented by a programmer such as 
playing or viewing interactions of certain dates, playing or viewing 
5 interactions about certain subjects, and so on. An editable software layer 
inherent to each viewer enables a programmer to build such constraints and 
selective functionality into a particular viewer, and to add the edited viewer 
to an IMA. 

Application Program Interfaces (API's) 229 are provided to allow a 

10 user to send obtained data or calculated results to connected peripheral 
devices or software modules or applications as may be required for 
operations such as printing, faxing, sending over a network, etc. If it is 
desired by enterprise rules, for example that a customer be able to print 
certain text or graphic information obtained through IMA 239, then the 

15 appropriate interface may be provided. 

Database Access Modules (DAM's) 231 are provided for allowing 
access to normally restricted databases that may be connected to CINOS 
architecture. Such databases will of course include multimedia information 
systems (MIS), customer information systems (CIS), which may also 

20 include contact and agent associated data, external databases such as may be 
hosted by the enterprise, and so on. Constraints may be applied to DAM's 
23 1 pertaining to which and or what portions of certain databases may be 
accessed by an application. 

Programming language modules 233 are provided and adapted to 

25 facilitate the type of platform/system that an IMA such as MA 239 will be 
created for. One IMA such as IMA 239 may be adapted to run on a variety 
of system types and platforms. System platform modules are provided as 
API's for intended supported system/platform combinations. Mathematical 
function modules (MFM's) are provided and adapted to interact with 
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CINOS databases for the purpose of performing pre-selected calculations 
such as cost averaging and so on. 

In this particular embodiment, IMA 239 is a finished application 
ready to be distributed. IMA 239 contains default display modules (not 
5 shown) for the purpose of enabling computer screen display on a user's 
system as is known in the art. IMA 239 may be stored in a special 
applications server (not shown) connected to the CINOS network either at 
WAN level or at the level of the hosting communication center. The 
method of distribution for MA's such as MA 239 may be of the form of a 

10 WEB-based client presentation to a user such as in customer window 135 of 
Fig. 5, for example. MA 239 may also be of the form of a browser plug-in 
accessible via a server such as may be the case with a special applications 
server as described above. In other instances, such applications may be 
made a part of an agent's desktop and so on. There are many and varied 

15 possibilities. 

In this particular embodiment, which is exemplary only, MA 239 is 
of the form of an interactive purchase guide for bulk paper products as 
illustrated via underlined title. MA 239, in this example, is logically 
separated into two distinct operations or functions. These are operation 241 

20 and operation 243. Operation 241 is a product preview interactive guide, 

while operation 243 is an order guide. The number of operations built in to 
an MA such as MA 239 will depend upon the intended purpose of the 
application according to enterprise rules. 

For exemplary purposes, assume that MA 239 which is, in this case, 

25 a purchase guide for bulk paper products, is to be presented to a corporate 
buyer who is new on the job. Because he is new, he may be uncomfortable 
with his own knowledge of how much or what kind of paper to buy. His 
predecessor may have a long purchase history with the enterprise. 
Therefore, he requests an MA such as MA 239 that will allow him to 
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preview products, browse the past purchase history of his predecessor, and 
perform a calculation that averages, by month, the last years paper purchases 
made by his company. 

According to enterprise rules, MA 239 adheres most closely to the 

5 buyer's request. That is, It allows for preview of products (241), and leads 
the buyer toward an order (243). MA 239 may, in some instances, be 
designed specifically for one buyer if it is determined that his level of 
business contribution warrants it. However, in most cases, MA 
applications such as application 239 will be more generic with interchange 

10 between different users accomplished with some editing performed based 
upon the intended use of the application and user parameters. 

A communication center may provide a number of standard MA's 
with each MA adapted to a different objective. A communication center 
may also provide custom-built MIAs for any specific purpose. A certain 

15 amount of editing ability renders one MA usable in more than one 
situation. 

Referring now to MA 239, as previously described, operations 241 
(product preview) and operation 243 (order guide) are available and related 
to purchasing bulk paper products. Operation 241 begins by presenting two 

20 different platform options from which a user may select. A platform A may 
be a Windows platform, and a platform B may be a UNIX platform. There 
may be more or fewer options regarding platforms. Similarly, applicable 
modules such as may be generic to a certain platform are installed with each 
platform. In this way, one application may be run on differing platforms. 

25 An API, labeled as such, shown logically connected beneath 

platforms A and B is illustrative of an interface for linked modules 
depended on platform choice. A database module (DAM) is first logically 
connected to the API module previously described. The DAM controls 
which database or databases may be accessed by MA 239, A window 
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shown immediately beneath the DAM provides an edit interface wherein the 
author or programmer may insert additional constraints, such as allowing 
access to only certain database sections and so on. 

A mathematical function module (MFM) labeled as such is shown 
5 beneath, and logically connected to the edit window. MFM is adapted to 
allow prescribed mathematical operations to be performed relative to 
database information such as cost averaging, grouping by product 
preference, and so on. Various modules as have been described herein may 
bring up additional displays on a user's computer if the module in question 

10 offers a choice of operation or returns readable results. Furthermore, 

standard preview modules (not shown) may be presented as object models 
and invoke standard viewers such as may be installed on the user's 
computer system. 

An interactive media viewer block (IMV) shown logically connected 

15 to MFM, allows the user to view pre-selected media interactions that are 

persistently stored in a database such as MIS 189 of Fig. 7. The IMV block 
shown may represent a plurality of unique IMVs or a single IMV. In this 
case three IMVs are involved, and these are represented by the blocks 
labeled TXT (text), VBD (video), and AU (audio). Each individual MV has 

20 an edit layer wherein a programmer may apply limitations or constraints 
relative to viewing capability. In some cases the same limitations may be 
applied to all the IMVs of an application in one editing sequence, such as by 
doing one edit and copying that edit to other IMVs. Although there are 
three illustrated viewer modules that make up the IMV in this example, 

25 more or fewer viewer modules may be used depending upon the intended 
use of MA 239 and enterprise rules. 

Although not explicitly shown, each IMV is editable through a 
software layer. In this way, a user may be limited to viewing certain media 
interactions and transactions that are allowed via enterprise rules. For 
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example, TXT viewer may only be able to view e-mails from the user and 
agent in a specific interaction thread, but not intermittent e-mails on the 
thread that may be from agent to agent or supervisor to agent and so on. 
Because each interactor with CINOS has an identification, and all 

5 interactions from or to them are so identified, these identifiers may be used 
in the edit layer of each viewer to constrain the user. In this way, a user may 
be granted access to a history database and view only his interactions 
without imposing on other users who share the system. Likewise, agents or 
supervisors charged with the task of reviewing the activities of certain other 

10 agents may use applications such as MA 239, adapted for the stated 

purpose, and be constrained in terms of whose interactions (agent's) may be 
viewed, and so on. In this manner full use may be provided to specialized 
users without exposing otherwise sensitive information that is not pertinent 
to the user or the purpose of the MA. 

15 Operation 243 created to allow a user to place an order for products 

is, in this case, a logical close for the previous operation. A module labeled 
Media Options may present standard media choices that the enterprise 
accepts for placing an order such as DP phone, e-mail, and so on. A 
connected text module (TXT) allows the user to send a quick text order 

20 while on-line. A send button sends a completed order to the enterprise. 

It will be apparent to one with skill in the art that an MA such as 
MA 239 may be programmed for virtually any enterprise objective without 
departing from the spirit and scope of the present invention, such as those 
already described. By utilizing pre-built modules instead of writing codes 

25 line-by-line, programmers may greatly increase the efficiency of application 
preparation and presentation to users. In many cases only slight editing is 
required to present a new application to a particular user. By using COM, 
and other known conventions such as Java, applications are quickly 
assembled or modified as has been taught herein. 
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Fig. 1 1 is a process flowchart illustrating logical steps for building 
an IMA for a user interacting with CENTOS according to an embodiment of 
the present invention. The process described below is meant to be just one 
example of many differing processes that may be implemented when 
5 building and customizing an IMA such as MA 239 of Fig. 10. The process 
components and order of which they are assembled will depend largely 
upon the type and purpose of the application being assembled and enterprise 
rules. 

In step 238, the programmer or application author opens his tool kit. 

10 Such a tool kit may be part of tool kit 125 of Fig. 4 on a CINOS desktop 

interface. In step 240, the author sets system and platform parameters. That 
is, he inserts the proper functional modules for use of the application on 
specific platforms and or system types. For example, the author may set up 
one application to run on more than one platform or system such as IBM 

15 and Macintosh. It should be noted here that if more than one type of system 
is supported in one application, then the associated modules will need to be 
included as well. 

In step 242, the author selects a programming language module 
containing libraries of known programming languages and codes. As 

20 known in the art, there are different programming languages used for 

different platforms. The author, in addition to building with pre-assembled 
modules, or building blocks, may have to write certain functional code in 
the supported language. In a preferred embodiment, the author has access to 
these language libraries from within his tool kit. 

25 In step 245, database access module(s) (DAMs) are selected and 

inserted. As previously described with reference to Fig. 10, these modules 
will determine which and what portions of databases may be accessed. In 
an alternative embodiment these restrictions may also be a part of the 
editable layer of MVs. Step 247 covers mathematical functions relative to 
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selected databases. Mathematical function modules (MFM's) allow a user 
to perform pre-defined operations. MFMs may or may not be needed in an 
IMA. This step may be omitted if no such functions are requested by a user 
or otherwise required in an application. 

In step 249, interactive media viewers (IMV's) are created using 
viewer modules adapted to view certain media of the type that stored 
interactions comprise. An IMV is a module that may comprise one or more 
than one media viewer. Each of these viewer modules are editable (via 
software layer) and may function alone or as a component of a larger 
module (comprising more than one viewer). 

In step 251, closing modules are inserted to complete the 
application. For example, order modules are one example of a closing 
modules. Modules adapted to return displayed results would be another 
example. Moreover, peripheral device API's may be inserted to allow 
results to be printed, faxed, sent over the network, etc. In this way, a 
supervisor reviewing the performance of a group of agents may report to 
other concerned parties such as managers, enterprise board members, or the 
like. 

The example as illustrated herein is basic but is deemed adequate by 
the inventor for illustration of one typical IMA building sequence. The 
description and order of steps may vary considerably. 

MA's such as MA 239 are transportable over a network and may 
be stored on a special applications server at network level, or within the 
communication center. In some embodiments, user's will be connected to 
the Internet when using IMA's allowing CINOS access. In other 
embodiments, agents may access CINOS resources while working off-line 
with respect to the Internet. In such cases, logging on to CINOS is still 
required. 
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It will be apparent to one with skill in the art that IMA 239 as taught 
herein is interactive and displayable on a PC/VDU that is logged into 
CINOS through a WAN. However, this is not specifically required to 
practice the present invention, but rather preferred. Other embodiments 
5 may include presenting a CTI interface such as an IVR wherein a user may 
interact with the application via voice or touch tone response. 

In still another embodiment, such applications may be presented via 
external media such as on a floppy disk(s) or CD-ROM wherein a user may, 
by inserting the disk or CD, obtain the ability of accessing the enterprise via 
10 WAN, gaining access to CINOS, and performing an objective with the 
MA. 

It will be apparent to one with skill in the art that CINOS may be 
implemented in a single communication center, or in a plurality of 
communication centers linked via WAN without departing from the spirit 

15 and scope of the present invention. 

It will also be apparent to one with skill in the art that rules may be 
created which govern access to CINOS without departing from the spirit and 
scope of the present invention. For example, customers may be required to 
subscribe to CINOS, and may also be provided with a customer application 

20 enabling such access. In another embodiment, access may be given to the 
general public according to established security rules governing commerce, 
financial transactions, and other processes. 

There are many existing and future implementation opportunities for 
an interaction operating system such as CINOS many of which have already 

25 been stated. The spirit and scope of the present invention is limited only by 
the claim that follow. 
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What is claimed is: 

1. In an object-oriented programming interface for use by a programmer, a 
5 software Interactive Media Viewer (IMV) module, comprising: 

a code set adapted to access and render media code from multimedia 
files stored in a data repository; and 

an editable layer allowing the programmer to program selective 
control of access by the IMV to the multimedia files. 

10 

2. The IMV of claim 1 wherein the IMV further comprises one or more 
software interfaces to other software modules that may be grouped in an 
Interactive Media Application (IMA) with one or more IMVs. 

15 3. An IMV as in claim 1 wherein the IMV is adapted to access and render 
multimedia code of one type. 

4. An IMV as in claim 1 wherein the MV is adapted to access and render 
multimedia code of more than one type. 

20 

5. The MV of claim 1 wherein the multimedia files stored in the data 
repository represent multimedia transactions, and are characterized with tags 
according to one or more of date, time, participants, file type, company 
affiliation of participants, subject or issue, and relationship to other 

25 multimedia files, and wherein MVs are limited through the editable layer to 
tags of the multimedia files. 
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6. A programming application for creating an Interactive Multimedia 
Application (IMA) which includes access to and rendering of multimedia 
files stored in a data repository, comprising: 

first selectable software modules providing functionality for an 
5 Interactive Multimedia Application other than access to and rendering of the 
multimedia files; and 

at least one selectable Interactive Multimedia Viewer (IMV) 
software module including a code set adapted to access and render media 
code from multimedia files stored in a data repository and an editable layer 
10 allowing the programmer to program selective control of access by the IMV 
to the multimedia files; 

wherein by selecting, including, and editing software modules the 
programmer is enabled to create the IMA. 

15 7. The programming application of claim 6 wherein the IMV further 

comprises one or more software interfaces to the first selectable software 
modules. 

8. A programming application as in claim 6 wherein the IMV is adapted to 
20 access and render multimedia code of only one type. 

9. A programming application as in claim 6 wherein the MV is adapted to 
access and render multimedia code of more than one type. 

25 10. The programming application of claim 6 wherein the multimedia files 
stored in the data repository represent multimedia transactions, and are 
characterized with tags according to one or more of date, time, participants, 
file type, company affiliation of participants, subject or issue, and 
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relationship to other multimedia files, and wherein IMVs are limited 
through the editable layer to tags of the multimedia files. 

1 1. A multimedia communication center, comprising: 
5 an access interface for outside communication; 

an interface to communication center personnel; 
a storage system for recording multimedia transactions in a data 
repository, the stored transactions characterized by tags representing one or 
more of date, time, participants, file type, company affiliation of 
10 participants, subject or issue, and relationship to other multimedia files; and 
A programming application for creating an Interactive Multimedia 
Application (IMA) which includes access to and rendering of the 
multimedia files stored in the data repository; 

wherein the programming application is characterized by first 
15 selectable software modules providing functionality for an Interactive 
Multimedia Application other than access to and rendering of the 
multimedia files, and at least one selectable Interactive Multimedia Viewer 
(IMV) software module including a code set adapted to access and render 
media code from multimedia files stored in a data repository and an editable 
20 layer allowing the programmer to program selective control of access access 
by the IMV to the multimedia files, wherein selecting, including, and 
editing software modules the programmer is enabled to create the IMA. 

12. The multimedia communication center of claim 1 1 wherein the IMV 
25 further comprises one or more software interfaces to the first selectable 

software modules. 



13. A multimedia communication center as in claim 1 1 wherein the IMV is 
adapted to access and render multimedia code of only one type. 



-59- 



14. A multimedia communication center as in claim 1 1 wherein the IMV is 
adapted to access and render multimedia code of more than one type. 

5 15. A multimedia communication center as in claim 1 1 wherein the 
multimedia files stored in the data repository represent multimedia 
transactions, and are characterized with tags according to one or more of 
date, time, participants, file type, company affiliation of participants, subject 
or issue, and relationship to other multimedia files, and wherein MVs are 

10 limited through the editable layer to tags of the multimedia files. 

16. In a Multimedia Communication Center environment which includes 
access to and rendering of multimedia files stored in a data repository, a 
method for assembling an Interactive Multimedia Application (IMA), 

15 comprising steps of: 

selecting first selectable software modules providing functionality 
for an Interactive Multimedia Application other than access to and rendering 
of the multimedia files; 

selecting at least one selectable Interactive Multimedia Viewer 
20 (IMV) software module including a code set adapted to access and render 

media code from multimedia files stored in a data repository and an editable 
layer allowing the programmer to program limitations limiting access by the 
IMV to preselected media files; 

editing the editable layer of the at least one IMV; and 
25 joining the selected and edited modules to form the IMA. 

17. The method of claim 16 wherein the IMV further comprises one or 
more software interfaces to the first selectable software modules. 
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18. The method of claim 16 wherein the MV is adapted to access and 
render multimedia code of only one type. 

19. The method of claim 16 wherein the IMV is adapted to access and 
5 render multimedia code of more than one type. 

20. The method of claim 16 wherein the multimedia files stored in the data 
repository represent multimedia transactions, and are characterized with tags 
according to one or more of date, time, participants, file type, company 

10 affiliation of participants, subject or issue, and relationship to other 

multimedia files, and wherein IMVs are limited through the editable layer to 
tags of the multimedia files. 
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Abstract of the Disclosure 

A programming application for creating an Interactive Multimedia 

5 Application (IMA) which includes access to and rendering of multimedia 
files stored in a data repository has first selectable software modules 
providing functionality for an Interactive Multimedia Application other than 
access to and rendering of the multimedia files, and at least one selectable 
Interactive Multimedia Viewer (IMV) software module including a code set 

10 adapted to access and render media code from multimedia files stored in a 
data repository and an editable layer allowing the programmer to program 
limitations limiting access by the IMV to preselected media files. In the 
programming application a programmer, by selecting, including, and editing 
software modules the programmer is enabled to create the IMA. IVRs may 

15 access and render code of one multimedia type or more than one type, and 
are adapted to interact with other software modules in an application. 
Multimedia files stored in the data repository represent multimedia 
transactions, and are typically characterized with tags according to one or 
more of date, time, participants, file type, company affiliation of 

20 participants, subject or issue, and relationship to other multimedia files, and 
IMVs are limited through the editable layer to tags of the multimedia files. 
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